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The  Honorable  Jack  Brooks 

Chairman,  Committee  on  Government  Operations 
House  of  Representatives 

Dear  Mr.  Chairman: 


Subject: 
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Review  of  a  Computer  Hardware  Acquisition  for  the 
National  Oceanic  and  Atmospheric  Administration  in 
Boulder,  Colorado  (AFMD-81-92) 


D 
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In  your  January  22,  1981,  letter  (end.  I)  you  asked  us  to  re¬ 
view  the  recent  acquisition  of  computer  hardware  for  the  National 
Oceanic  and  Atmospheric  Administration  (NOAA)  in  Boulder,  Colorado. 
Your  committee  had  received  information  that  the  purchase  was  un¬ 
warranted  because  (1)  the  justification  data  were  manipulated  and 

(2)  the  present  computers  were  underutilized  and  mismanaged. 

In  discussion  with  your  office  we  agreed  to  address  these 
points : 

(1)  The  computer  hardware  procurement  history  of  NOAA's 
Environmental  Research  Laboratories  (ERL)  and  whether 
ERL  was  following  a  pattern  of  noncompetitive  acquisi¬ 
tions. 

(2)  The  justification  for  the  recent  computer  hardware  ac¬ 
quisition  for  the  Boulder  computer  center. 

(3)  The  management  of  the  Boulder  computer  center. 

(4)  The  bid  protest  by  the  third  party  maintenance  vendor, 
Computer  Maintenance  Technologies,  Inc. 

(5)  ERL's  planning  for  a  fiscal  1982  competitive  procurement. 


We  briefed  your  office  on  the  results  of  our  review  and  were 
requested  to  furnish  a  report  on  our  findings.  In  summary,  we 
found:  . 

rJ)l^ The  Boulder  computer  center  is  not  following  a  pattern  of 


0) 

noncompetitive  acquisitions^' 

( 2 j  ^The  recent  sole-source  procurement  of  computer  hardware  ap-f  ^ 
7  pears  to  be  justified  as  an  interim  bridge  to  a  fully  com-  '  If 
petitive  procurement  in  1982 J  '  3 
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Boulder  computer  center  does  not  appear  to  be  mismanaged^' 


(H)  V"The  bid  protest  by  Computer  Maintenance  Technologies,  Inc., 
was  rejected  by  the  Department  of  Commerce,  and  the  company 
chose  not  to  pursue  the  protest  further  with  us^ • 

( 5/  —We  have  serious  concerns  about  the  adequacy  of  ERL's  plan¬ 
ning  for  their  future  computer  resource  needs.,.. 

Because  of  our  concerns  about  the  adequacy  of  ERL's  planning 
for  their  future  computer  resource  needs,  your  committee  may  wish 
to  consider  asking  the  Department  of  Commerce  to  defer  procurement 
actions  pending  completion  of  our  review  of  the  adequacy  of  ERL's 
computer  resources  management  plan. 


OBJECTIVES,  SCOPE,  AND  METHODOLOGY 

Since  the  objective  of  our  review  was  to  provide  information 
on  the  points  raised  by  your  committee,  we  concentrated  on  those 
areas.  We  made  extensive  use  of  information  gathered  at  the  Boulder 
computer  center  as  part  of  our  1980  review  of  NOAA  automatic  data 
processing  (ADP)  planning.  1/ 

To  determine  the  Boulder  computer  center's  procurement  history 
we  interviewed  ERL  and  NOAA  headquarters  personnel  responsible  for 
ADP  planning.  We  also  examined  relevant  documents,  including  in¬ 
ternal  ERL  correspondence  as  well  as  communications  between  ERL, 
NOAA,  and  the  Department  of  Commerce. 

We  evaluated  the  justification  for  the  recent  computer  hard¬ 
ware  acquisition  by  concentrating  on  the  merit  of  the  information 
received  by  your  committee  indicating  the  purchase  was  unwarranted. 
This  evaluation  included  two  interviews  with  the  source  of  the  ad¬ 
verse  information.  We  also  interviewed  ERL  technical  and  manage¬ 
ment  personnel  at  the  Boulder  computer  center  as  well  as  an  onsite 
vendor  software  special is'  . 

In  addition,  we  reviewed  pertinent  documents,  including 
computer-generated  utilization  reports  used  by  the  computer  center 
for  billing  and  accounting  both  before  and  after  the  recent  pro¬ 
curement.  We  reviewed  computer  center  reliability  and  performance- 
related  documents  as  well  as  internal  ERL  correspondence  and  memo¬ 
randums  regarding  the  acquisition. 


Our  review  of  the  management  of  the  Boulder  computer  center 
was  based  on  interviews  with  ERL  headquarters  and  computer  center 
management  personnel.  We  obtained  additional  information  on  com¬ 
puter  center  management  practices  and  staff  attitudes,  and  reviewed  O 
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internal  ERL  memorandums  from  users  regarding  computer  center  man¬ 
agement. 

Our  evaluation  of  the  adequacy  of  ERL  planning  for  its  1982 
competitive  procurement  was  based  primarily  on  a  review  of  ERL  in¬ 
ternal  documents  and  correspondence  including  computer  requirements 
justifications  prepared  by  the  laboratories.  We  also  conducted 
interviews  with  ERL  personnel  responsible  for  computer  planning. 

ERL  PROCUREMENT  HISTORY 


We  found  that  ERL  has  not  been  following  a  pattern  of  non¬ 
competitive  sole-source  procurement.  ERL  has,  until  the  recent 
sole-source  acquisition  of  the  Control  Data  Corporation  (CDC)  Cyber 
170/750  for  the  Boulder  computer  center,  satisfied  its  general 
purpose  computing  needs  by  competitively  procuring  computer  sys¬ 
tems  or  by  obtaining  excess  computer  systems  from  other  Government 
agencies.  ERL  also  purchases  computer  time  from  universities  and 
government  facilities. 

The  Boulder  computer  center  is  one  of  four  general  purpose 
computing  centers  operated  by  ERL.  The  Boulder  center  is  the  only 
one  of  the  four  that  is  shared  among  more  than  one  laboratory.  The 
other  three  computer  centers  are: 

—Geophysical  Fluid  Dynamics  Laboratory  in  Princeton,  New 
Jersey,  which  operates  a  competitively  procured  Texas 
Instruments  Advanced  Scientific  Computer,  now  being  replaced 
by  a  competitively  procured  CDC  Cyber  205. 

—National  Severe  Storms  Laboratory  in  Norman,  Oklahoma,  which 
operates  a  Systems  Engineering  Laboratories  8600  computer 
obtained  as  excess  equipment  from  another  Government  agency 
in  1972. 

—Atlantic  Oceanographic  and  Meteorological  Laboratory  in 
Miami,  Florida,  which  operates  a  Univac  1108  computer  ob¬ 
tained  as  excess  equipment  in  1979.  This  laboratory  also 
makes  extensive  use  of  the  Boulder  computer  center  as  well 
as  purchasing  time  on  the  Air  Force  Weapons  Laboratory 
Cray-1  computer  in  Albuquerque,  New  Mexico. 

The  Boulder  computer  center  operates  in  a  service  center  en¬ 
vironment  providing  computing  services  to  13  ERL  components  in  and 
out  of  Boulder,  the  National  Bureau  of  Standards  Boulder  Labora¬ 
tories,  the  National  Telecommunications  and  Information  Adminis¬ 
tration  Boulder  Laboratories,  the  Environmental  Data  and  Informa¬ 
tion  Service  in  Boulder,  and  to  a  small  number  of  other  Government 
agencies  and  Government  contractors  in  the  Boulder  area.  The 
Boulder  computer  center  has  performed  seven  computer  system  pro¬ 
curements  since  1961.  Although  it  has  stayed  with  the  Control  Data 
Corporation  product  line  for  most  of  its  history,  the  current  sole- 
source  procurement  is  the  only  one  of  that  type.  The  others 
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consisted  o£  three  competitive  procurements  and  three  obtained  from 
the  General  Services  Administration  excess  list.  In  1967,  one  of 
its  competitively  procured  computers  was  upgraded  by  the  vendor  to 
a  more  powerful  computer  system. 

RECENT  ACQUISITION  OF  COMPUTER  HARDWARE 
APPEARS  TO  BE  JUSTIFIED 

The  recent  sole-source  procurement  of  computer  hardware  to 
replace  two  aging,  unreliable  computers  at  the  Boulder  computer 
center  appears  to  be  justified  as  an  interim  procurement,  although 
the  interim  computer  hardware  is  currently  underutilized.  Appli¬ 
cable  procurement  regulations  for  the  interim  procurement  were 
followed  and  nothing  we  found  substantiated  the  allegation  that 
the  justification  data  for  the  procurement  were  manipulated. 

Applicable  procurement  regulations 
were  followed  in  justifying 
the  interim  procurement 

The  General  Services  Administration  on  March  14,  1980,  pro¬ 
vided  guidance  to  its  Office  of  Agency  Services  and  Procurement 
for  the  handling  of  agency  procurement  requests  involving  augmen¬ 
tation  or  replacement  of  installed  AOP  equipment.  This  guidance 
was  issued  pending  clarification  of  existing  procedures,  including 
the  removal  of  the  interim  upgrade  1/  provisions  in  Federal  Prop¬ 
erty  Management  Regulation  101-35. 206 ( c) ( 4 ) .  According  to  that 
provision,  now  rescinded,  if  equipment  for  an  interim  upgrade  is 
acquired  noncompetitively,  the  agency  should  replace  the  equipment 
through  a  competitive  procurement  within  2  years  of  the  initial 
acquisition.  2/  The  guidance  treats  agency  procurement  requests 
for  sole-source,  specific  make  and  model,  or  compatible  equipment 
on  an  individual  basis,  rather  than  as  previously  prescribed  in 
the  regulation. 

The  General  Services  Administration  granted  ERL  a  delegation 
of  procurement  authority  in  September  1980.  ERL's  procurement  re¬ 
quest  indicated  that  it  would  competitively  replace  the  sole-source 
computer  and  estimated  the  system  life  at  3  years. 


1/Interim  upgrade  means  the  acquisition  of  additional  and/or  aug¬ 
mentation  of  installed  components  or  subsystems  of  AOP  equipment 
or  systems  on  a  temporary  basis,  pending  a  fully  competitive  re¬ 
procurement,  to  meet  unforeseen,  urgent,  data  processing  require¬ 
ments. 

2/On  Jan.  5,  1981,  the  General  Services  Administration  published 
a  revised  Federal  Property  Management  Regulation  101-35.2  which 
deleted  this  requirement.  See  46  F.R.  1213-1219,  Jan.  5,  1981. 
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ERL's  justification,  prepared  in  accordance  with  this  guidance, 
was  based  on  the  following  factors: 

—Data  processing  was  essential  to  the  agency  mission  (95  per¬ 
cent  of  ERL's  computer  usage  is  scientific  processing  in 
support  of  its  research  and  development  mission)  and  there 
existed  an  urgent  need  to  replace  its  aging,  unreliable  com¬ 
puter  systems. 

—The  procurement  was  necessary  for  economy  and  efficiency 
(scientists'  time  estimated  at  $675,000  per  year  was  being 
wasted  by  slow  turnaround  time  and  inadequate  processing 
capability) . 

—A  required  conversion  cost  study  prepared  by  ERL  and  ap¬ 
proved  by  the  Federal  Conversion  Support  Center  showed  it 
would  cost  $9.9  million  to  convert  to  a  noncompatible  sys¬ 
tem. 

ERL  further  justified  this  sole-source  acquisition  as  a  bridge  sys¬ 
tem  until  action  can  begin  on  ERL's  long  range  plan  for  a  series 
of  competitive  acquisitions  beginning  in  fiscal  1982.  NOAA  has 
fully  supported  a  budget  initiative  for  fiscal  1982  to  support  this 
long  range  plan. 

Justification  data  does  not  appear 
to  have  been  manipulated 

Your  committee  received  information  stating  that  the  sole- 
source  procurement  may  have  been  unwarranted,  in  part  because  the 
justification  data  had-  been  manipulated.  A  key  point  in  the  infor¬ 
mation  provided  to  the  committee  was  that: 

"NOAA/ERL's  computer  operations  management  was  fraudu¬ 
lent  in  its  omission  of  the  data  that  would  have  shown 
one  of  the  two  CDC  6600  computers  spending  61  percent 
of  its  'total  CPU  usage  time'  running  'do  nothing' 
maintenance  diagnostic  programs  during  the  Jan/March 
80  time  period  used  in  NOAA's  justification." 

The  Department  of  Commerce  justification  material  furnished 
to  the  General  Services  Administration  requesting  a  delegation  of 
procurement  authority  did  not  use  the  data  cited  by  the  committee 
source. 

Computer  center  records  substantiate  that  during  the  January 
thru  March  1980  time  period  cited,  one  CDC  6600  spent  61  percent 
of  its  total  processing  time  running  a  category  that  includes  main¬ 
tenance  diagnostic  programs.  According  to  computer  center  offi¬ 
cials,  the  high  percentage  was  because  this  second  central  process¬ 
ing  unit  was  recently  installed  and  was  still  having  technical 
problems.  These  officials  and  a  Control  Data  system  software 
analyst  stated  it  was  common  practice  when  operating  with  6600s, 
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which  lack  basic  error  detection  capabilities  such  as  parity  check¬ 
ing,  to  run  maintenance  diagnostic  routines  concurrently  with  user 
jobs  when  experiencing  hardware  problems.  They  said  the  newly  in¬ 
stalled  unit  could  be  making  errors  affecting  user  output  and,  if 
the  maintenance  programs  were  not  running,  the  operator* and  users 
might  be  unaware  of  the  problem. 

During  the  January  thru  March  period  cited,  the  newly  installed 
central  processing  unit  was  in  a  test  mode  with  its  usage  not  be¬ 
ing  billed  to  users.  After  the  system  became  operational  in  July, 
users  were  billed  and  the  percentage  of  time  spent  running  main¬ 
tenance  diagnostics  dropped  substantially. 

Present  computer  system  is  underutilized 

The  replacement  CDC  Cyber  170/750  system  is  underutilised. 

This  underutilization  resulted  from  an  underestimation  of  its 
capability.  ERL  used  several  sources  to  determine  that  the  Cyber 
170/750  has  approximately  2.25  to  4.00  times  the  computing  power 
of  a  single  CDC  €600  computer  system.  In  choosing  the  Cyber 
170/750  to  replace  its  two  CDC  6600  computer  systems,  ERL  selected 
the  lower  figure  of  2.25.  However,  because  of  the  nature  of  ERL's 
workload,  the  Cyber  170/750' s  computing  capacity  is  on  the  higher 
end  of  the  range.  ERL  had  estimated  that  the  computer  would  be 
saturated  in  June  1981;  however,  because  of  the  underestimation 
of  computer  capacity  this  has  not  occurred.  ERL  computer  usage 
data  for  July  1981  indicated  that  the  central  processing  unit  was 
busy  36  percent  of  the  time.  Accordingly,  ERL  is  now  selling  ex¬ 
cess  time  to  other  Government  agencies  and  considering  some  modi¬ 
fication  of  its  1982  competitive  procurement  plan. 

COMPOTER  CENTER  DOES  NOT  APPEAR 
TO  BE  MISMANAGED 

In  our  opinion,  the  Boulder  computer  center  does  not  appear 
to  be  mismanaged.  Since  early  1980,  the  current  computer  center 
management  has  taken  a  number  of  positive  steps  to  improve  the 
center's  operations.  These  steps  include: 

—Better  control  of  operational  procedures  through  daily  op¬ 
erational  review  meetings. 

—Acting  to  foster  future  competition  by  moving  the  computer 
center  to  standard  off-the-shelf  system  software. 

—  Improving  user  utilization  of  the  computer  system  by  estab¬ 
lishing  user-oriented  training  courses. 

COMPOTER  MAINTENANCE  TECHNOLOGIES.  INC.. 

DID  NOT  PURSUE  BID  PROTEST 

Computer  Maintenance  Technologies,  Inc.  (CMTI),  filed  bid  pro¬ 
tests  with  the  Department  of  Commerce  and  us  on  December  2  and  3, 
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1980,  respectively.  The  bid  protests  involved  a  clause  in  ERL's 
contract  with  Control  Data  Corporation  that  provided: 

"The  Government  shall  take  every  precaution  to  prevent 

the  unauthorized  duplication  or  disclosure  of  Software 

Programs.  Authorization  for  duplication  may  be  obtained 

only  from  the  Contractor 

CMTI  stated  it  had  been  their  experience  that  Control  Data 
Corporation  would  not  allow  third  party  maintenance  vendors  access 
to  computers  using  CDC  software  nor  to  the  software  listings.  CMTI 
stated  such  access  was  sometimes  necessary  to  facilitate  tne  repair 
of  Government-owned  computer  peripheral  equipment  for  which  CMTI 
still  had  maintenance  responsibility.  They  requested  a  ruling  on 
this  clause  so  that  third  party  maintenance  vendors,  including  CMTI, 
could  continue  to  successfully  bid  on  maintenance  contracts  for 
Government-owned  Control  Data  equipment. 

The  Department  of  Commerce  reported  that  it  is  a  basic  prin¬ 
ciple  in  Federal  procurement  that  when  the  Government  buys  a  pro¬ 
prietary  commercial  product  developed  at  private  expense  it  does 
not  ootain  unlimited  rights  to  publicly  disclose  the  operation  of 
that  product  unless  those  unlimited  rights  are  specifically  agreed 
to  in  the  contract,  We  agree  with  this  determination. 

In  its  report  on  the  protest  the  Department  of  Commerce  deter¬ 
mined  the  contract  clause  questioned  was  proper  and  recommended 
that  we  deny  the  protest.  CMTI  did  not  respond  to  our  request  as 
to  whether  it  wanted  to  pursue  the  protest  further.  Accordingly, 
we  consider  the  protest  closed. 

GAO  CONCERN  WITH  ERL  1982 
COMPETITIVE  PROCUREMENT  PLANNING 

ERL  has  prepared  a  long  term  plan  for  meeting  its  automatic 
data  processing  requirements  for  fiscal  1982  through  1988.  NQAA 
has  approved  this  plan  and  is  supporting  the  related  budget  re¬ 
quests  of  $2.9  million  a  year  for  7  years  ($20.3  million)  begin¬ 
ning  in  fiscal  1982.  The  plan  envisions  a  competitive  procurement 
of  a  mixture  of  seven  mid-range  systems  to  process  small  joDS  plus 
a  single  large  system  to  process  large  computer  modeling  jobs. 

we  are  seriously  concerned  that  ERL's  computing  requirements 
have  not  been  properly  determined.  ERL  laboratories,  which  lack 
ADP  planning  expertise,  were  tasked  with  determining  their  future 
computing  requirements  with  only  minimal  guidance.  Consequently, 

ERL  developed  its  requirements  in  system  terms,  with  an  emphasis 
on  workload  and  hardware,  rather  than  user  requirement  terms.  User 
requirements  are  more  closely  related  to  tne  functions  to  oe  per- 
f o  rmed . 

In  our  opinion,  user  requirements  must  be  clearly  and  accu¬ 
rately  defined  if  they  are  to  oe  effective  in  tne  resource  planning 
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process.  They  must  also  be  described  with  service  levels  in  mind, 
for  instance,  timeliness,  accuracy,  cost,  and  reliability  of  the 
data  processing  function. 

User  requirements  should  reflect  the  functional  needs  of  the 
user,  stated  in  terms  the  user  understands  (data  volume,  response 
l  time,  average  record  length)  and  not  system  requirements  (memory, 

central  processing  unit  time,  tape)  which  define  the  computing  en- 
.  vironment.  In  user  requirements,  quantities  are  stated  in  units 

that  measure  user  tasks,  not  in  computer  system  terminology.  The 
accuracy  of  computer  resource  selection  is  contingent  upon  a  com- 
I  prehensive  understanding  of  user  requirements. 

Because  computing  requirements  form  the  basis  for  proper  sys¬ 
tems  selection,  it  would  be  very  easy  to  end  up  with  systems  that 
do  not  meet  ERL's  real  needs.  To  reduce  this  risk,  your  committee 
may  wish  to  consider  requesting  the  Department  of  Commerce  to  defer 
procurement  actions  until  we  have  reviewed  the  adequacy  of  ERL's 
computer  resources  management  plan. 


At  your  request,  we  did  not  obtain  comments  on  the  matters 
,  discussed  in  this  report. 

As  agreed  with  your  office,  we  plan  no  further  distribution 
of  this  report  until  30  days  from  its  date.  At  that  time,  we  will 
send  a  copy  to  the  Director  of  the  Environmental  Research  Labora¬ 
tories  and  make  copies  available  to  other  interested  parties. 

Sincerely  yours, 

i 


of  the  United  States 


Comptroller  General 


Enclosure 
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CongrtM  si  tip  fitnixb  ^istn 
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January  22,  1981 


Honorable  Elmer  8.  Staats 
Comptroller  General  of  the  United  States 
Washington,  O.C.  20548 

Dear  General : 

Information  of  a  serious  nature  has  been  forwarded 
to  the  Committee  concerning  the  acquisition  of  computer 
hardware  for  the  National  Oceanic  and  Atmospheric 
Administration  (NOAA)  of  the  Department  of  Commerce 
at  3oulder,  Colorado.  The  information  indicates  that 
a  recent  procurement  was  unwarranted  because  (1)  the 
justification  data  was  manipulated,  and  (2)  the  present 
computers  are  underuti 1 ized  and  mismanaged. 


Notwithstanding  the  pending  legal  determination  of 
the  bid  protest  filed  by  Computer  Maintenance  Technologies, 
Inc.  on  Oecember  3,  1980,  I  request  that  GAO  undertake 
a  review  of  this  computer  acquisition.  A  determination 
should  be  made  if  the  acquisition  was,  indeed,  justified 
and  whether  the  NOAA  is  managing  its  computer  center 
at  8oulder  properly.  I  would  also  like  GAO  to  include 
its  evaluation  of  the  protest  in  its  report.  Dan  Soranno 
of  the  Conmunity  and  Economic  Development  Division  has 
done  similar  ADP  reviews  for  the  Committee  in  the  past 
and  it  would  be  helpful  it  he  could  be  assigned  this 
work. 

With  best  wishes,  I  am 
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